Displaying image data from a scanner capsule

ABSTRACT

An ingestible image scanning pill captures high resolution images of the GI tract as it passes through. Images communicated externally have exact location determination. Image processing software discards duplicate information and stitches images together, line scan by line scan, to replicate a complete GI tract as if it were stretched out in a straight line. A fully linear image is displayed to a medical professional as if the GI tract had been stretched in a straight line, cut open, laid flat out on a bench for viewing—all without making any incisions in a live patient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/500,232, filed Jul. 9, 2009, which claims the benefit of U.S.Provisional Application No. 61/079,342, filed Jul. 9, 2008, each ofwhich is incorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to medical diagnostics using aningestible medical diagnostic device, i.e. pill endoscopy.

2. Background Art

Endoscopes are commonly used by physicians to obtain images of internaltissues and organs as a diagnostic tool. Typically an endoscope is usedto probe from a patient's mouth down into the upper gastro intestinal(GI) tract. During a colonoscopy, endoscopes are used to probe from theanus up into the lower GI tract. An endoscope is essentially a tube witha light and camera at its tip. Images can be transmitted outside thepatient's body either optically (fiber optic cable), or converted by acamera to a digital signal and sent by wire up the endoscope and into anelectronic device outside the patient.

Images presented to the physician are seen from the point of viewdescribed, i.e. looking down a tube. As a result of a complex folding ofthe GI tract and in combination with a fairly short distance for imagecapture due to low levels of lighting and/or low resolutions at fardistances, only a short section of the GI tract can be viewed at anygiven time during an invasive procedure based on the location of theendoscope.

The population of the United States is aging. The first wave of the 78million “Baby Boomers” is beginning to turn 60 years old. Coincidingwith this aging of population is a rising concern regarding the publichealth, and a generally more educated patient in technology awareness.There has been an explosion in diabetes cases, estimated at 194 millioncases worldwide today, and predicted to be 350 million cases by year2025. Obesity currently affects two thirds of the U.S. population. Thereis a rising incidence of cardiac problems for women (the #1 cause ofdeath for women). Hepatitis C will soon reach epidemic levels, infectingnearly 5 million people, more than the number of approximately 1.2million people infected with HIV/AIDS in the U.S. Celiac disease affectsapproximately 3 million people in the U.S., with about 97% beingundiagnosed. The prevalence of further serious conditions, such ascancer, ultra- or ulcerative-colitis, lactose intolerance, allergies,etc., indicate that there is a need for simple and easy diagnostictechniques, especially because many of these diseases are chronic,requiring repeat testing over time. Some conditions, such as cancer, aremost responsive to treatment if caught in the early stages. Cancer, forexample, is best detected in the digestive tract. Given that cancerousgrowth can occur in as little as one to two years, it is essential todetect cancer or cancerous precursors at least annually, or preferablybiannually. Physician and health care resources are currently alreadystretched and will fail if the current technology, process and procedureare not altered to suit the needs of the baby boomer market of the nearfuture. Time-saving and simple solutions to testing are needed.

The current population desires speedy testing and fast answers to theirhealth questions. Many current testing and monitoring systems arelimited by old technology and processes that take days, if not weeks,for results. These test methods, if not inconvenient and potentiallyembarrassing, are at least in most cases intrinsically painful or riskyto patients.

One ingestible diagnostic device in the market today is a disposable RFcamera pill or capsule camera, which captures images of the digestivetract as it passes through. Current camera pill usage by patients andphysicians is limited for several reasons. First and foremost, currenttechnology is very large in comparison to most ingestible medicines andnutritional supplements. The excessive size is in part a result of theselection of power-inefficient communication methods. The large sizemandates pre-screening of patients (an additional process,inconvenience, and cost). The large size also leads to a reasonably highpotential that the device can become lodged within the GI tract. Thismay lead to a highly invasive surgical removal requirement, whichcarries all the risks associated with some surgeries.

Conventional RF camera pills require a bulky reading device worn as abelt around the waist and adhesive sensors attached to the body tocapture an electromagnetically-coupled signal transmitted from the pill.The patient is required to report to a physician's office forprescreening, to initiate use of the camera pill, and to be fitted withthe belt reader. The belt reader is worn for 24 hours, during which timethe camera pill captures images and transmits the images to the readerbelt. At the end of a diagnosis period, the patient (and belt reader)must return to the physician. The physician downloads images from thebelt reader and analyzes the images. The physician may analyze theimages and discuss the results with the patient at yet anotherappointment during a subsequent visit. Thus, current RF camera pillsrequire at least two trips to the physician's office, as well as thewearing of a cumbersome belt reader with leads attached to the skin.

This diagnostic process is both inconvenient and uncomfortable. It alsocarries a risk of surgical removal, due to the size of the currentcamera pills. Current technology does not offer a recorded positionwithin the body associated to the specific image taken. Physicians mustachieve a location of an image of interest through yet anotherprocedure. Furthermore, the current camera pills are expensive devices,and are resorted to when other bowel disease diagnostic techniques, suchas endoscopy and colonoscopy (each of which are extremely intrusive),present results that need farther investigation. Further, theelectromagnetic signals used to transmit the images may harm thesensitive tissue they pass through in order to be detected outside thebody. Therefore, the current ingestible camera pill has significantdeficiencies.

Current technology RF camera pills attempt to mimic the imaging carriedout by physicians using laparoscopes and endoscopes. The camera pillilluminates the GI tract as it passes through and takes pictures atregular intervals, much like frames of a movie. The physician laterviews a series of images in a format of a movie that when paused appearmuch like images provided by an endoscope. Due to the natural movementof the digestive tract, the resultant movies from the current camerapills depict spurts of forward and backward movements that are awkwardand not reviewer friendly, leading to issues of reviewer focus andoverall system effectiveness.

What is needed is a way to display a high resolution image captured by anew generation of ingestible image scanning pills to a medicalprofessional in a manner that allows the professional to easily find,zoom in on and get context for any abnormalities that may be observed.

BRIEF SUMMARY OF THE INVENTION

This section is for the purpose of summarizing some aspects of thepresent invention and to briefly introduce some preferred embodiments.Simplifications or omissions may be made to avoid obscuring the purposeof the section. Such simplifications or omissions are not intended tolimit the scope of the present invention. Consistent with the principlesof the present invention as embodied and broadly described herein, thepresent invention includes an ingestible image scanning pill which isable to capture high resolution images of the wall of the GI tract as itpasses through it propelled by peristaltic action. The peristalticaction produces a forward and backward, churning motion as an aid to thedigestive process. Images of the GI tract are captured by “scanning”line by line and region by region the GI tract as the pill moves throughit. Images are obtained not by “photographing” as in many of the knowntechnologies, but rather by “scanning” line by line and area by area.The GI tract can be illuminated by various types of sources includingwhite light, multi-spectrum light, narrow spectrum light, infra-red,ultra-violet, and even non-light energies such as, for example,acoustical energy, etc.

Images communicated outside of the patient represent tissues at exactlocations determined based on signals transmitted. Such images can becommunicated by radio wave, optically (such as, for example, using anoptical fiber), by acoustic signals, etc. Signals representing imagesare received outside the patient's body and are processed by one or morecomputers running software capable of discarding duplicate informationand stitching together, line scan by line scan a complete GI tract as ifit were stretched out in a straight line. The processed, fully linearimage is then displayed to a medical professional as if the GI tract hadbeen stretched into a straight line, cut open, laid flat out on a benchfor viewing—all without making any incisions in a live patient, as easyas swallowing a pill.

The image processing software is capable of concurrently renderingdifferent aspects on the GI tract, similar to topology views created bycomputer software of terrain as would be viewed from the top or ‘flownthrough’ from the side to side. Aspects from a dissection viewpoint(laid out on a table), to a close-up dissection view, to a generatedlooking down the tube viewpoint.

Alternative form factors to a “pill” can also be used, such as, forexample, a modified endoscope or modified catheter.

Further features and advantages of the invention, as well as thestructure and operation of various embodiments of the present invention,are described in detail below with reference to the accompanyingdrawings. It is noted that the invention is not limited to the specificembodiments described herein. Such embodiments are presented herein forillustrative purposes only. Additional embodiments will be apparent topersons skilled in the relevant art(s) based on the teachings containedherein.

BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES

The accompanying drawings, which are incorporated herein and form a partof the specification, illustrate the present invention and, togetherwith the description, further serve to explain the principles of theinvention and to enable a person skilled in the pertinent art to makeand use the invention.

FIG. 1 shows a partial view of a human 102 according to an embodiment ofthe present invention.

FIG. 2 shows an example block diagram of ingestible capsule 104,according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a communications module according to anembodiment of the invention.

FIG. 4 shows a view of ingestible capsule 104, with communicationsmodule 204 including acoustic communications module 302.

FIG. 5 is a schematic diagram of an example sensor communicationsnetwork.

FIG. 6 is a schematic illustrating how ingestible capsule 104 may alsocommunicate with computing device 108 via an intermediate sensor linkmodule 602.

FIG. 7 is a schematic diagram illustrating how a sensor link module 602may be configured in various ways.

FIG. 8 depicts a system for display utilizing multiple computerprocessors each connected directly to a monitor for display.

FIG. 9 depicts a system for display utilizing multiple monitorsconnected to a single processor for display.

FIG. 10 depicts a single display and single processor with multiplewindows each with a display of information.

FIG. 11 depicts information for display for an image scanner capsule

FIG. 12 depicts a normalized human subject and mapping of GI tractwithin.

FIG. 13 depicts a display of an ingestible capsule path as it transits aGI tract with several positional controls.

FIG. 14 depicts a projection of an ingestible capsule path and icons forareas of concern upon a patient's body.

FIG. 15 is a flow chart of a main processing system of image processingsoftware for rendering the displays depicted in FIG. 11.

FIG. 16 is a flow chart of a subroutine of the image processing softwarefor rendering a linear aspect of the displays depicted in FIG. 11.

FIG. 17 is a flow chart of a subroutine of image processing software forrendering a drawing of the full GI tract in 2D.

FIG. 18 is a flow chart of subroutines of image processing software forupdating aspects of the displays and providing annotations as depictedin FIG. 11.

FIG. 19 includes flow charts of subroutines of the image processingsoftware for updating a tract aspect and updating time and distance,respectively, of the displays depicted in FIG. 11.

FIG. 20 is a flow chart of a subroutine of image processing software forupdating a zoom aspect.

FIG. 21 is a flow chart of subroutines of image processing software forupdating a tube aspect.

FIG. 22 is a flow chart of an overall scanned image collection,processing, and reporting system.

FIG. 23 is a detailed flow chart of a scanned image creation process.

FIG. 24 is an exemplary depiction of scanned data corresponding to FIG.23 process.

Features and advantages of the present invention will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings, in which like reference charactersidentify corresponding elements throughout. In the drawings, likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements. The drawing in which an elementfirst appears is indicated by the leftmost digit(s) in the correspondingreference number.

DETAILED DESCRIPTION OF THE INVENTION Introduction

The invention will be better understood from the following descriptionsof various “embodiments” of the invention. Thus, specific “embodiments”are views of the invention, but each does not itself represent the wholeinvention. In many cases individual elements from one particularembodiment may be substituted for different elements in anotherembodiment carrying out a similar or corresponding function. It is to beappreciated that the Detailed Description section, and not the Summaryand Abstract sections, is intended to be used to interpret the claims.The Summary and Abstract sections can set forth one or more but not allexemplary embodiments of the present invention as contemplated by theinventor(s), and thus, are not intended to limit the present inventionand the appended claims in any way.

The arrangements and techniques described herein are particularlysuitable for improved imaging using an ingestible diagnostic pill,although they are applicable to other devices, such as for example,laparoscopes and endoscopes.

An ingestible image scanning pill captures high resolution images of theGI tract as it passes through. Examples of such scanning pills aredescribed in U.S. patent application Ser. No. 11/851,221, filed Sep. 6,2007, titled “Ingestible Low Power Sensor Device and System forCommunicating with Same,” and U.S. Provisional Patent Application No.61/028,102, filed Feb. 12, 2008, titled, “Ingestible Endoscopic OpticalScanning Device,” each of which is incorporated by reference herein inits entirety. Images communicated externally have exact locationdetermination. Example techniques for locating a diagnostic pill are setforth in U.S. patent application Ser. No. 11/851,179, filed Sep. 6,2007, titled “Imaging and Locating Systems and Methods for a SwallowableSensor Device, which is incorporated by reference herein in itsentirety. Image processing software discards duplicate information andstitches images together, line scan by line scan a complete GI tract asif it were stretched out in a straight line. Stitching can be completedduring the scanning process (real time) or alternatively can be batchprocessed after all scan information is collected, or completed througha periodic batch process (pseudo-real time). After a full image or apseudo-real time partial image is available, automated image analysisfunctions will each insert their results into a database with an indexinto the available image and offset within the image. A fully linearimage with optional automated analysis results is displayed to a medicalprofessional as if the GI tract had been stretched in a straight line,cut open, laid flat out on a bench for viewing and optionally withsuspect abnormalities indicated—all without making any incisions in alive patient. The medical professional will review the image andsuspected areas of abnormality and also insert their own suspected areasand/or general comments into a database similar to that of the automatedanalyses. Finally, the medical professional will optionally create areport of findings with, among other details and comments, a selectionof suspected abnormalities and their corresponding images at the sameaspect of their review. A system may be automated so as to automaticallygenerate alerts when stages are complete, such as when a full image isavailable, when a report of findings is complete, or when automatedimage analysis is completed.

The invention is described in terms of specific embodiments that eachincorporate certain features of the invention. The embodiments merelyexemplify the invention. It is not intended that each embodiment includeall features of the invention. The scope of the invention is not limitedto the disclosed embodiments. The invention is defined by the claimsappended hereto.

References in the specification to “one embodiment,” “an embodiment,”“an example embodiment,” etc., indicate that the embodiment describedmay include a particular feature, structure, or characteristic, butevery embodiment may not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to effect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

Furthermore, it should be understood that spatial descriptions (e.g.,“above,” “below,” “up,” “left,” “right,” “down,” “top,” “bottom,”“vertical,” “horizontal,” etc.) used herein are for purposes ofillustration only, and that practical implementations of the structuresdescribed herein can be spatially arranged in any orientation or manner.Likewise, particular bit values of “0” or “1” (and representativevoltage values) are used in illustrative examples provided herein torepresent data for purposes of illustration only. Data described hereincan be represented by either bit value (or by alternative voltagevalues), and embodiments described herein can be configured to operateon either bit value (or any representative voltage value), as would beunderstood by persons skilled in the relevant art(s).

The example embodiments described herein are provided for illustrativepurposes, and are not limiting. Further structural and operationalembodiments, including modifications/alterations, will become apparentto persons skilled in the relevant art(s) from the teachings herein.

Ingestible Diagnostic Pill

The embodiments described herein are set forth in the context of aningestible diagnostic pill. The following provides general explanationabout the configuration and arrangements of ingestible diagnostic pillssuitable tor making use of the inventions described herein.

The example embodiments of an ingestible diagnostic pill describedherein are provided for illustrative purposes, and are not limiting.Further structural and operational embodiments, includingmodifications/alterations, will become apparent to persons skilled inthe relevant art(s) from the teachings herein.

Structures and methods for an ingestible diagnostic pill are described.An ingestible diagnostic pill is also referred to as an “ingestiblecapsule” because of its generally capsule shape. It is also referred toan “ingestible pill” or “diagnostic pill.” The ingestible diagnosticpill may be swallowed by a human (or animal) to diagnose or aid in thediagnosis of one or more conditions through either an immediatedetection or a historical and/or statistical analysis of multipledetections of conditions or attributes over a time period. Exampleembodiments are described below as related to a human subject, forillustrative purposes. However, embodiments of the present invention areapplicable to animals other than humans, including livestock (cattle,sheep, pigs, chickens, turkeys, ostriches, etc.), pets (e.g., dogs,cats, horses, etc.), and other animals of interest such as race horsesor other performance/sport animals. Such applicability to these types ofanimals, and other types, will be apparent to persons skilled in therelevant art(s) from the teachings herein, and is within the scope andspirit of embodiments of the present invention.

Furthermore, example embodiments are described below as related topassing an ingestible capsule through a gastrointestinal tract, forillustrative purposes. However, embodiments of the present invention areapplicable to further bodily systems other than the gastrointestinaltract, including the circulatory system, the urinary tract, and otherbodily systems and additionally other means of entry or implant into abody cavity of an animal or human. Such applicability to other types ofbodily systems will be apparent to persons skilled in the relevantart(s) from the teachings herein, and is within the scope and spirit ofembodiments of the invention.

FIG. 1 shows a partial view of a human 102 according to an embodiment ofthe present invention. In FIG. 1, human 102 has swallowed or ingested aningestible capsule 104. Ingestible capsule 104 is configured to senseone or more attributes or conditions of human 102 as ingestible capsule104 passes through human 102. While passing through human 102,ingestible capsule 104 transmits information in a communication signal106 to, be received on the outside of the human 102. Ingestible capsule104 may send information to and receive information from an externaldevice, via communication signal 110, or it may be a beacon that onlyemits information to the external device. As shown in FIG. 1, anexternal computing device 108 may receive communication signal 106.Computing device 108 may be used to display the information received incommunication signal 106, to interact with the information, to processthe information, and/or to transmit the information (raw or processed)to another entity or component. In an embodiment, computing device 108can interact with ingestible capsule 104 to control functions ofingestible capsule 104.

In embodiments, human 102 may be provided with one or more ingestiblecapsules 104 that human 102 may at designated times and/or periodicallyswallow to perform an analysis of one or more health-related conditionsof human 102. Multiple ingestible capsules 104 may interact with device108 and/or each other.

FIG. 2 shows an example block diagram of ingestible capsule 104,according to an embodiment of the present invention. In FIG. 2,ingestible capsule 104 includes an acoustically transmissiveencapsulation 208 that holds one or more sensors 202, a communicationsmodule 204, and a power source 206. Although FIG. 2 illustratesingestible capsule 104 as having three sensors 202 a, 202 b, and 202 c,one of skill in the art will recognize that any number of sensors may beincluded in ingestible capsule 104. In one embodiment, there may be nosensor(s) 202 at all, providing a capability to track the pill movementin space, hence allowing a mapping of a gastro-intestinal tract and alsothe time of movement within that tract.

In an embodiment were ingestible capsule 104 has one or more sensor(s)202, sensor(s) 202 are used to sense (e.g., measure, detect, etc.) areceived stimulus 210, and generate a sensor output signal 212. Sensoroutput signal 212 may be a digital or analog signal, depending on theparticular implementation of sensor 202. In alternative embodiments theacoustically transmissive encapsulation 208 may be made of sensor(s)202, or sensor 202 may be integrated within the materials known asacoustically transmissive encapsulation 208. Ingestible capsule 104 caninclude any number of sensors 202, each of which may all sense the samecondition or may sense a different condition than another sensor 202.Sensor 202 may detect and/or interact directly with conditions of thebody. Sensor 202 may also detect and/or interact with signals emanatingfrom the pill and reflecting off nearby tissues, such as is the casewith, for example and without limitation, a camera or optical scannerdetecting light that originates from the capsule, ultrasonic detectors,and radioactivity sensors. In an embodiment, sensor 202 detectsreflections of signal 106 from nearby gastro-intestinal and other bodytissues.

Logic control 214 initiates activity of sensor 202 via controlconnection 211. Sensor 202 detects or interacts with the body andproduces a sensor output signal 212. Communications module 204 receivessensor output signal 212, and generates communication signal 106 toinclude information based on sensor output signal 212. Communicationsignal 106 is transmitted from ingestible capsule 104.

In an example embodiment, as shown in FIG. 3, communications module 204may include an acoustic communications module 302, configured totransmit and/or receive an acoustic communications signal. For example,acoustic communications module 302 may include an acoustic transmitter.Sensor output signal 212 is modulated on an acoustic signal that istransmitted as communications signal 106 by the acoustic transmitter.The acoustic communications signal 106 may be transmitted by radiatingelement 304, which may be, for example, an electromechanical transduceror piezoelectric (e.g., PZT, PVDF, etc.) element or transducer thatvibrates at acoustic frequencies. An example acoustic frequency range inwhich acoustic communication signal 106 may be transmitted is 20 Hz to 3MHz, although the frequency may be an acoustic frequency higher or lowerthan this range in some applications. An example frequency for acousticcommunications signal 106 is 2 MHz. In a likewise fashion, acousticcommunications module 302 may include an ultrasonic communicationsmodule, configured to transmit and/or receive a communications signal atultrasonic frequencies (e.g., greater than 20 KHz). Communicationsmodule 204 may be configured to modulate information of sensor outputsignal 212 according to a variety of modulation techniques, includingamplitude modulation (AM), frequency modulation (FM), and phasemodulation (PM), and including any combination of these modulationtechniques, including in quadrature modulation schemes. Acousticpressures according to embodiments may have various levels, includinggreater or lower than 1 Pa, including in the KPa (or greater) range tothe μPa (or less) range.

FIG. 4 shows a view of ingestible capsule 104, with communicationsmodule 204 including acoustic communications module 302. In FIG. 4,communications module 204 is coupled to acoustically transmissiveencapsulation 208. Acoustically transmissive encapsulation 208 vibratesaccording to acoustic communications module 302 to transmit acommunications signal 402, which is an acoustic version ofcommunications signal 106. In FIG. 4, acoustically transmissiveencapsulation 208 functions as an acoustic radiating element, vibratingat acoustic frequencies according to acoustic communications module 302.

Returning to FIG. 2, operation of ingestible capsule 104 may be gatedand controlled by control logic 214, which itself may be operating in asub-threshold voltage (Vt) manner (e.g., to save power), or controllogic 214 may operate in normal bias modes. In an embodiment, ingestiblecapsule 104 is an autonomous device with one way communication(transmission capability), so that control logic 214 may be extremelysimple, and thus would not consume much power even when operating innormal bias modes. However, in another embodiment, ingestible capsule104 may communicate in both directions, and may be configured to receiveinstructions from computing device 108. Control logic 214 may thus haveadditional complexity in order to, for example, decode and implementreceived instructions.

Power source 206 provides power (e.g., via electrical energy) to operatethe components of ingestible capsule 104 that require power, such ascommunications module 204 and/or sensor 202. Power source 206 mayinclude, for example and without limitation, a battery, a liquid, or anenergy harvesting module.

In an embodiment, ingestible capsule 104 is configured for low poweroperation, including extreme low power (XLP) operation. To achieve XLPoperation, ingestible capsule 104 can use one or both of a very smallbattery and energy harvesting to operate ingestible capsule 104. In anembodiment, circuits of ingestible capsule 104 are implemented in one ormore integrated circuits (ICs), in a technology such as CMOS, or othertechnology. The IC(s) and any other internal components of ingestiblecapsule 104 may be mounted to a circuit board, or mounted directly toacoustically transmissive encapsulation 208. Thus, in embodiments, powersource 206 is configured for low power output, including supplying powerin the milliwatt and microwatt ranges. Such low power requirementsenable the size of power source 206 to be minimal.

In a CMOS embodiment, MOSFET circuits may be configured to operate in adeep sub-threshold voltage (sub-Vt) mode, which lowers their switchingtime to acoustic switching frequencies, and lowers their powerconsumption, by orders of magnitude. In such a mode the MOSFET devicesoperate as analog devices. Such operation was demonstrated in themid-1980's by Carver Meade with regard to eye and ear chips. Such a modeof operation eliminates the need for digitizing the sensor information,which can be very power intensive, and which further reduces the powerconsumption by a large factor.

Acoustically transmissive encapsulation 208 contains sensor 202,communications module 204, and power source 206, and is configured to beingestible by or inserted within a human and/or animal. Acousticallytransmissive encapsulation 208 may be the size of a vitamin or othertype of pill that is ingestible by humans. For example, acousticallytransmissive encapsulation 208 may be approximately 3 mm in diameter andapproximately 5 mm in length. Acoustically transmissive encapsulation208 may be any suitable shape, including oval, elliptical (as shown inFIG. 2), capsule shaped, or spherical. The small size of acousticallytransmissive encapsulation 208 allows ingestible capsule 104 to beeasily ingested by an average human 102. Further, the small size ofacoustically transmissive encapsulation 208 increases the ability ofingestible capsule 104 to pass completely through the digestive systemof a human 102 without becoming trapped due to size incompatibility.

Acoustically transmissive encapsulation 208 may be made from a varietyof non-digestible or slow rate of digestion materials, including: aplastic material, such as a resin, a resinoid, a polymer, a cellulosederivative, a casein material, and/or a protein; a metal, including acombination of metals/alloy; a glass material; a ceramic; a compositematerial; and/or other material/combination of materials. In aparticular embodiment, acoustically transmissive encapsulation 208 maybe comprised of a material that aids in the sensing of biological,chemical, or other attributes of body material that touches or comes inclose proximity to the acoustically transmissive encapsulation 208, suchas could be called an integrated encapsulation and sensor material.

After being swallowed by human 102, ingestible capsule 104 eventuallypasses from human 102, such as when human 102 has a bowel movement toexcrete waste. In an embodiment, ingestible capsule 104 is disposable.In another embodiment, ingestible capsule 104 may be recovered, (andrecycled) for reuse.

Depending upon the ability or control of the patient, ingestible capsule104 may alternatively be inserted into a lower gastrointestinal tract ofhuman 102 as a suppository device.

Depending on the configuration of sensor 202, while passing throughhuman 102, ingestible capsule 104 can sense conditions and/or featuresof any part of the gastrointestinal tract, and any of thematerials/fluids contained within and/or secreted by the organs in thegastrointestinal tract or organs indirectly associated with thegastrointestinal tract. Ingestible capsule 104 can also receiveconditions or signals from even more remote body organs such as acousticpickup of heartbeat and/or breathing and more indirect conditions suchas temperature. In an embodiment, a camera or an optical scanningimaging system is coupled to ingestible capsule 104 to allow visualobservation of human 102.

As mentioned, ingestible capsule 104 transmits information incommunication signal 106 to be received outside human 102, such as bycomputing device 108. In an embodiment, computing device 108 may beconfigured to communicate with a remote entity 502, such as shown in anexample sensor communications network 500 shown in FIG. 5. Computingdevice 108 may be configured to communicate with remote entity 502 usingwired and/or wireless links, in a direct fashion or through a network504. For example, computing device 108 transmits a communication signal506 to network 504, which transmits a communication signal 508 to remoteentity 502. Network 504 may be any type of network or combination ofnetworks, such as a telephone network (e.g., a land line and/or cellularnetwork), a personal area network (PAN), a local area network (LAN),and/or a wide area network (WAN) such as the Internet.

Remote entity 502 may be one or more of a variety of entities, includinga human and/or computer-based entity. For example, remote entity 502 mayinclude a diagnosing physician who receives information collected byingestible capsule 104 (and optionally processed by computer device 108)in communication signal 508.

As shown in FIG. 5, sensor communications network 500 may include areturn communications path from remote entity 502 through network 504 tocomputing device 108. For example, a return communication signal 510 istransmitted by remote entity 502 to network 504, which transmits areturn communication signal 512 to computing device 108. In this manner,remote entity 502 (e.g., diagnosing physician and/or computer system)can provide feedback to computing device 108 in communication signal 512regarding the analysis of human 102 performed by ingestible capsule 104.Return communication signal 512 may include any type of data/informationformat for providing the feedback, including an email, a text message, atext file, a document formatted for commercially available wordprocessing software, a proprietary document/data format, auditoryalarms, alerts and messages, etc.

Ingestible capsule 104 may also communicate with computing device 108via an intermediate sensor link module 602, as shown in FIG. 6. Sensorlink module 602 receives communication signal 106 from sensor 202.Sensor link module 602 transmits a communication signal 604 to computingdevice 108, to provide the information sensed by sensor 202 to computingdevice 108. For example, sensor link module 602 may be used wheningestible capsule 104 communicates using an acoustic communicationssignal having a power level too low to reliably be received by computingdevice 108. As shown in FIG. 6, sensor link module 602 is coupled tohuman 102.

In another embodiment, sensor link module 602 may provide acommunication interface between ingestible capsule 104 and network 504,such that a separate computing device 108 is not required. In such anembodiment, sensor link module 602 may perform functions of computingdevice 108 described above, and thus sensor link module 602 may bereferred to as a computing device.

Multiple sensor link modules 602 may provide a capability of locationdetection through triangulation and other algorithms, capable ofdetecting sensor device 104 to a very accurate, three (3) dimensionallocation within human 102. In ah embodiment, multiple sensor linkmodules 602 may be attached to human 102 at various locations in orderto receive the interior acoustic signal from different angles. Sensorlink module 602 may be, for example, directly attached to the skin ofhuman 102, such as by an adhesive or a strap. Sensor link module 602 maybe attached to human 102 in one or more locations, including the head,neck, chest, back, abdomen, arm, leg, etc. With regard to receivingcommunication signal 106 from ingestible capsule 104 passing through thegastrointestinal tract, ingestible capsule 104 may be attached to theneck, chest, back, and/or abdomen for a short signal path.

An amount of received information is proportional to the number ofsensor link modules 602 attached to human 102. The array of sensor linkmodules 602 may be attached at specific locations on human 102 toincrease, and even maximize, the received diagnostic information.Multiple sensor link modules 602 can identify a specific location of theingestible capsule which can be used for linking a location to thedetection of a sensed material. The location can also be used toidentify a historical analysis of the track taken by the ingestiblecapsule and the speed of passage.

For example, the attachment of an array of three or more sensor linkmodules 602 to human 102 may enable triangulation or other locationfinding algorithms to be used to locate ingestible capsule 104 in human102. Alternatively, one or more sensor link modules 602 having three ormore receivers each may be used to the same effect. By locatingingestible capsule 104 in human 102, a location of a sensed material inhuman 102 can be determined.

In embodiments, sensor link module 602 may be configured in variousways. For instance, FIG. 7 shows an example sensor link module 602,according to an embodiment of the present invention. As shown in FIG. 7,sensor link module 602 includes control logic 702, a sensorcommunication module 704, storage 706, a remote communication module708, and a power source 710.

Sensor communication module 704 receives communication signal 106 fromingestible capsule 104. Sensor communication module 704 demodulates thesensor-related information of communication signal 106. Furthermore,sensor communication module 704 may process and/or convert a format ofthe information received in communication signal 106. For example,sensor communication module 704 may perform an analog-to-digital (A/D)conversion of the received sensor information, and outputs a sensorinformation signal. The sensor information signal may be received bystorage 706 and/or by control logic 702.

Storage 706 is configured to store the sensor information of the sensorinformation signal. Storage 706 may include any type of suitablestorage, including a hard drive and/or memory devices. Storage 706 canoutput the stored information in a stored sensor information signal, forsubsequent transmission to computing device 108 by remote communicationmodule 708.

Control logic 702 is configured to control operation of sensor linkmodule 602.

Remote communication module 708 receives the stored sensor informationsignal, and formats the sensor-related information for transmission.Furthermore, remote communication module 708 transmits the sensorinformation in communication signal 604. Remote communication module 708may be configured to transmit communication signal 604 in a variety offormats/protocols, such as a standard RF communication protocolincluding Bluetooth, IEEE 802.11, Zigbee, or other communicationprotocol, standard or otherwise. For example, in embodiments, computingdevice 108 may be a Bluetooth, 802.11, and/or Zigbee configured handhelddevice such as cell phone, personal digital assistant (PDA), aBlackberry™, wrist watch, music player, or laptop, or other type ofcomputer, handheld, desktop, or otherwise. Remote communication module708 may also transmit an identification number assigned to ingestiblecapsule 104 for identification by a receiver.

Power source 710 provides power to elements of sensor link module 602that require power, such as control logic 702, sensor communicationmodule 704, storage 706, and remote communication module 708. Forexample, power source 710 may include one or more batteries that arerechargeable or non-rechargeable. Power source 710 may also (oralternatively) include an interface for externally supplied power, suchas standard A/C power.

As described above, in an embodiment, ingestible capsule 104 cantransmit an acoustic signal. By receiving the acoustic signaltransmitted by ingestible capsule 104, sensor link module 602 mayperform a type of ultrasound analysis based on the human interiorgenerated acoustic signal from ingestible capsule 104. As acousticcommunication signal 106 is transmitted through human 102 fromingestible capsule 104, signal 106 is transformed by attenuation,refraction, and reflection, as a function of the tissue of human 102that signal 106 passes through. The transformed signal thus providesadditional diagnostic information to sensor link module 602, very muchlike a diagnostic ultrasound conveys diagnostic information that can beanalyzed by a trained technician. The acoustic signal from ingestiblecapsule 104 may be viewed as an “interior” ultrasound or “sonogram”,which can be analyzed to extract additional diagnostic informationregarding human 102. In an embodiment, information received by sensorlink module 602 regarding the interior ultrasound signal can be used togenerate a graphical display of at least a portion of the interior ofhuman 102. An interior ultrasound can also be generated from an array ofPZT sensors configured around the circumference of the capsule, whichwould receive the ultrasound signal reflected by the tissue of the bodylumen. The captured ultrasound image data would then be sent from thecapsule to sensor link module 602.

Image Process

FIG. 22 depicts an overall system flow process for the acquisition,display, and diagnosis of image data from, for example, an ingestiblescanning capsule, according to an embodiment of the present invention.In the first process, 2210, image data is captured on the capsule andtransmitted to storage on a network resource as described in FIGS. 5 and6. In Step 2212, these single scans of 2210 are manipulated into asingle image. As could be expected, combining these multiple individualscans into a single image is quite complex, and a topic described below.Following this process, in step 2214, is an automated image analysisstep, which may trigger alerts to nurses, doctors, and other healthprofessionals in step 2216. Automated image analysis may look forpatterns, colors, transit distance, and even compare known imagelibraries for abnormalities. Step 2216 alerts may inform a healthprofessional that a capsule has entered a certain area, for instance, astomach, small bowel, or large bowel, or have completed passage of theentire digestion system. Additionally, step 2216 may send alerts as toemergency conditions such as bleeding, stoppage of the passage of thecapsule, or potentially critical findings of cancerous regions orsubstantially large foreign objects, as well as other alerts foroperations of equipment and medical diagnostic findings. Step 2218determines if the scan is a last scan of interest—and the last imagedata to be compiled into a single large image of the intestinal tract.Several conditions may trigger the last scan conclusion—a capsule out ofbattery indication, a loss of communication (capsule out of body, forexample), a distance traveled (either determined on the capsule, anadjacent receiver, or even a function on the network), an automatedprocess concluding the end of a desired region (for example, smallbowel).

Once a final inclusive scan is determined and received, a finalprocessing can occur upon the collective image. In step 2220,optionally, automated analysis is launched upon the full image.Functions can be identical to those in step 2214, however, functionssuch as total length, size, coloration, and a variety of comparison tothe whole will likely be accomplished after the last scan is received.The final automated analysis functions in step 2220 may includedetermination of the segments of the intestinal tract, such as stomach,small bowel, and large bowel, but may also include the numerous detailedparts known to medical professionals. Automated analysis functions willdetail findings as well as the location and aspect (zoom, colorfiltering, etc) within the final full image. For instance, a detectedcancer region may be more easily viewed and recognized at a zoomed out,general area image (akin to a city level on a Google map), and with nored colorization that might obscure the cancer tissue. In comparison, asanother example, evaluation of Celiac's disease, which erodes the villiof the intestinal wall, might be best evaluated with a close-up image(akin to Google's street level) in full color.

After each and/or all of a potential of multiple automatic analyses,notifications of the progress would be sent in step 2230. Alerts may bedirected to one or multiple recipients, such as medical professionalsfor evaluation of the imagery and analyses, doctor office staff forscheduling of equipment return, patient appointments, and schedulingfinal doctor review of the imagery & analyses, or even to the patientfor equipment return or to call in for an appointment. In step 2240,medical professionals will be presented with the resultant scannedimagery, results of automated or external analyses of the imagery, andwill have the opportunity to explore the imagery from differentmagnification levels, colorizations, and other aspects. Step 2240 isvery unique in presentation as compared with prior products and will bedetailed in a further section. In step 2250, reviewers may save commentson areas found to be of interest in either further evaluation and/orareas that appear similar enough to characteristics known ofirregularity by skilled medical professionals. Comments in step 2250would also save the particular aspect the medical professional iscurrently viewing, including, but not limited to zoom (magnificationlevel), colorization or color enhancements, tilt or rotation, and anyother configuration from the reviewer that would alter the visual imageleading a medical professional to make an assessment of an abnormality.A reviewer of step 2250 may wish to have a specialist or second opinionon a comment and would then indicate this within the system, which thenwould trigger step 2260, an automated alert to another reviewer. Anoriginal reviewer may exit and re-enter a system in step 2240, 2250, or2260 to view the results of said specialist review prior to moving to afinalization in step 2270.

A beginning of a report of findings is shown in step 2270, which is inpart selection of previous comments from their own analyses, analyses byother medical professionals or review of specialists, and also automatedanalyses comments. A final report may or may not include all commentsavailable, potentially only the most severe areas of abnormality mightbe of primary interest. Additionally, step 2270 may also save generalcomments as to the state of health of the patient, concerns orabnormalities not yet found but likely to occur in the future, and alsoabnormalities found and recommendations for future actions such as dietand/or treatments, and other items that would be obvious to medicalprofessionals. Upon completion of the selection of comments, a finalreport can be recorded and printed into a patient file in the last step,step 2280. Electronic Medical Records (EMR) are currently in frequentuse, and will likely be the repository of such reports, external to thesystem. However, an internal report logging history may be availablefrom within the system without the need of EMR. Since in large part thereport material is based upon quality images, it may be ideal to havepaper/printed reports available from a remote service with qualityprinting ability, and is so included in embodiments of this invention inthe combination of steps 2280 & 2290. After the completion of the finalreport, another opportunity for automated alerts is found in step 2290.Automated alerts expected from step 2290 would include an alert toschedule an appointment with the patient (with or without urgency), anindication to professional staff or service to print and file, a finalreport, and potentially alerts to fill out & submit therapeuticprescription or other medications, and other alerts that would beanticipated by medical professionals.

Scanned Image Creation

As one skilled in the art would recognize, a capsule being propelled byintestinal peristaltic action may produce scan data that may be capturedin a forward or backward position axially as related to a previous scandata, the capsule may be tilting with reference to the axis, the capsulemay be rotating about the axis, and the capsule may be adjacent totissue and also separated from tissue at any point in time and withinany scan. In addition, axial motion and acceleration are spurious, notconsistent as is the case with traditional scanning mechanisms likeflatbed scanners and satellite imagery. Thus, the general function ofcombining many pieces of scan data into one image is reasonably complexeven though in some part it does exist in a specific environment in thecombination of multiple satellite images into a single, larger image asdisplayed on the internet for services such as Google Earth and thelike.

FIG. 23 depicts an embodiment model for creation of a single image frommultiple scans taken over time and distance throughout the human gastrointestinal tract. This embodiment is based upon a scanning capsule asdefined in U.S. Provisional Patent Application No. 61/030,453, filedFeb. 21, 2008 and entitled “Radial Scanner Imaging System,” which isincorporated by reference herein in its entirety. Choice of a variety ofdifferent methods to produce a scanning capsule does not depart from thespirit and scope of this invention. FIG. 23 depicts image capture andprocessing that may occur on the capsule 104, the external computingdevice 108, or a device attached to network 504 such as remote entity404. Selection of a particular device for a certain processing step isanticipated and inherent in a design for power consumption. Otherselections of which device is used for processing does not depart fromthe spirit and scope of this invention.

In the embodiment of FIG. 23, a capsule starts by a single scan capture,as in step 2310. Several methods may be employed to determine when thisscan is to happen, such as for example a detection of motion, an elapseof time, or a combination thereof. Such determination is not an objectof this invention, but is described in U.S. Patent Application No.61/030,453. A single scan, as would be anticipated, if in normal visualperception colorization (RGB), would require illumination methods as alight source is not present within the human gastro-intestinal tract.Such illumination is not an object of this invention other than toacknowledge its necessity, existence, and required control of the lightsource. A single scan, thus, would consist of an illumination and acapture. Capture is accomplished through an imaging device such as aCMOS or CCD device. In one embodiment, a special configuration of astandard imaging device such as CMOS or CCD device may be applied. Inanother embodied, scanned ultrasound images are captured by an array ofPZT sensor elements.

An example of a side wall scanning capsule device is depicted in FIG.24. An intestine 2410 projects its image, onto a cone shaped mirror 2412which is angled and positioned to reflect the resultant image onto theimaging array 2414. A example scan, 2420, is shown as image 2422 uponarray 2414. Referring back to FIG. 23, step 2310 refers to the captureof this image 2422 by a device 2414, which may or may not be rectangularand may or may not be fully populated with image cells. Additionally,then step 2320 is a process by which a ring image is stored inelectronic elements such as memory cells, in which a depiction presentedto a human would result in an isosceles trapezoid. In FIG. 24, the ringimage 2422, with an arbitrary position 2424, results in an isoscelestrapezoid strip which is a linear projection of the ring similar to aflat map of the otherwise ball shaped earth. In FIG. 24, the imagemanipulation process is shown as 2430, and the resultant trapezoid isdepicted as 2435. Image 2435 is shown as an isosceles trapezoid forunderstanding only. Actual memory storage may be different, however, thedistance around the inner ring of image 2422 is a shorter path than theoutside ring and the differing sizes considered.

Referring back to FIG. 23, process step 2325 (also process 2440 in FIG.24) converts this difference of path lengths for multiple lines(depiction trapezoid) and converts the lines to equal lengths in aresultant rectangular depiction 2445 in FIG. 24. Since the originalcapsule is propelled by the peristaltic action of a human, the capsulemay rotate axially as well as tilt/yaw with respect to an axis of aproximal human intestine. FIG. 23 then depicts a process formanipulation of an image strip to adjust, or compensate for axialrotation (process 2330) and also tilt and yaw (process 2335).Additionally, as a larger, full image is being built, it may be appendedin one direction by single scans. Since a human intestine on averagemoves from stomach to colon, the image is generally appended in thisdirection. However, well known to a gastro-enterologist, a humanintestine will both propel forward (toward the colon) and backward(toward the stomach) a capsule at any given time, or scan. Thus, anadditional process, 2340, would be used to essential invert a resultantsingle scan if a direction of movement is backward. Alternatively, step2340 may discard completely reverse movement scans. It is important tonote that step 2330, 2335, and 2340, also shown as 2450 in FIG. 24 areseparated and ordered by way of example only. Steps 2330, 2335, and 2340may be done in any order, or may also be combined into a single complexoperation (2450) without departure from the spirit and scope of thisinvention. The resulting single scan of process steps through andincluding 2340 is depicted as 2455 in FIG. 24.

The single scan is now prepared to be merged with other such scans intoa larger, full image; the process shown as a first step 2345, whichdetermines at what specific location and rotation a single scan shouldbe placed into the fuller image. In a simple merge, a new scan is justappended upon the end of the larger image with no overlap or adjustment.Typically, however, it is best to have some overlap of an image todetermine exact merge points. Features present in images may not line upexactly, in which an additional process in the merge is employed toaverage out differences in features. A merge process could be as simpleas an arithmetic average of a pixel in a larger image with a pixel in asingle scanned image for all pixels in the single scanned image area.However, it is likely that a weighted averaging and other mathematicalfunctions would be deployed in a merge of a larger, fall image and asmaller single scanned image. FIG. 23 shows the step 2350 as a mergeprocess to include all varieties of merging functions and algorithms.FIG. 24 shows the merge process as step 2460 and the resultant depictionof the data as image 2465, a compilation of multiple merge processes ofmultiple single scanned images. The same process may be applied toscanned ultrasound images.

A compilation of images may be desired to be forwarded from a smallerdevice to a larger storage capacity device. In step 2360, a device mayelect to forward a full image (compilation of multiple single scans).Reasons may vary widely depending upon a particular implementation. Someexemplary reasons would be time elapsed (to achieve a pseudo real timeenvironment while also reducing image data transmitted), and also imagebuffer full, or a combination of both. If there is no reason to forwarda compilation, then process returns to capturing a next single scan.However, if a full image is to be forwarded, then this image will bequeued for transmission, prior to returning process flow to collect anext single image.

Image Display

Referring back to FIG. 22 and for review, a single image is availableafter step 2212. Auto analysis can be applied to this image in step2214. This process can be repeated many times based upon timing, amountof image data collected, or other analyses to image and data received(such as location). Repeating the process in a very timely manner wouldresult in a pseudo real time image update & availability. It isanticipated that a last scan, shown in step 2218, triggered by a lastcommunication, an exit of the capsule from the body, a determinationbased upon auto analysis in step 2214, or other system applications andtimers will make available a full and final image for an automated andmedical professional review. Final automated analysis will occur in step2220, and upon completion(s) will trigger a single or multiple alerts toa single or multiple persons or systems that the image and results ofanalysis are both complete and available, as is depicted in step 2230.

It is an object of embodiments of this invention to apply potentiallymany discrete automated processes selected either by medicalprofessionals, general subscription profiles, or another method ofpre-selection of automated processes. Each discrete automated processwill analyze the image and produce results via comments with respect toa particular section (location & magnification), aspect, andcolorization of the image. A future review by a medical professionalwill be able to view an abnormality at the image detail (location, zoom,aspect, colorization) that was selected as an abnormality of interest byan automated process. Embodiments of this invention anticipate automatedprocesses individually or in combination drawing upon multiple imagelibraries of exemplary abnormalities to be evaluated for diagnosis, withconfirmation by a medical professional. Furthermore, these imagedatabases would contain multiple example images for each abnormality orpathology of interest. Automated processes then would compare images ofa capsule endoscopy with multiple images of a library of abnormalities.The automated process would select a matching area (location, zoom,colorization, aspect) of the capsule endoscopy and comment upon thematched abnormality, the area (e.g., location, zoom, colorization, andaspect), and the likelihood of a match (percentage or other numbersystem is anticipated).

Furthermore, it is an object of embodiments of this invention that therewould be automated processes with computational imagery analysis for thegeneration of the location and likelihood of particular common sectionsof the gastro-intestinal tract, for example the beginning and end of asmall bowel. Automated processes are anticipated to be constructed as:i) imagery analysis without image library comparison, ii) directcomparison with a particular image of a library with a current image ofcapsule endoscopy, and iii) imagery analysis by way of comparisonbetween a current capsule endoscopy and a fully characterized andcollective set of images composing an entire library of images. Use ofany and/or all of the above methods of imagery analysis is anticipatedand an object of embodiments of the present invention. Additionally,computational image analysis would allow for extraction from image data,three dimensional (3-D) views and “super-resolution”. Scanned image datais ideally suited to apply computational analysis techniques that can'tconveniently be achieved by conventional photography.

FIG. 22, step 2240 depicts another form of availability of an analysisby other reviewers of images prior to a primary medical professionalreview, in, the automated fashion of work flow management by way ofalerts and availability of images for review. By way of example only, aprimary care physician might pre-select a trusted gastro-enterologistand a trusted celiac disease specialist to review results of a patient'scapsule endoscopy as soon as it is available for review. These trustedspecialists may be of personal contact within a local community, butalso anticipated is a service, or services, that offer remote reviewingprofessionals as a solicited service to the primary care physician. Thelatter service is similar in nature to a current x-ray process at somehospitals, whereby an x-ray, digitized, is transported electronically toreviewers often in India, and rapidly interpreted with results returnedto the hospital. It is then anticipated that specialized services forthe interpretation of capsule endoscopy images will be automaticallypreselected by a primary medical professional or staff thereofassociated with a particular patient, and that the pre-selection of theservice will occur prior to the analysis of the imagery by the sameprimary medical professional or staff thereof.

It is an object of embodiments of this invention to also afford theopportunity of an initial analysis of a capsule endoscopy imagery by aprimary medical professional and then a subsequent electronic request ofa service (new or past used) and a further alert system generation uponthis referred service completion of analysis prior to the primarymedical professional's report of finality, depicted in step 2280.Embodiments of the present invention are generally constructed tofacilitate automated processing of capsule endoscopy imagery in asystematic and preselected method while also allowing a more timeconsuming and manual method to a primary medical professional on a caseby case basis as is sometimes necessary for proper evaluation ofnon-typical patient symptoms and ailments. In step 2240, the selectedadditional (not primary) analysis conclusions are documented withrespect to a patient's capsule endoscopy image. As is with the automatedimage analyses of step 2220, these analyses are in the form of commentswith respect to a specific location, magnification, colorization, andaspect so as to afford a repeatable view to a primary medicalprofessional with the results of the analyses. In step 2250, each ofpotentially many additional reviewers of the image save the results oftheir analyses with respect to the image. In step 2260, automatedalerts, upon completion of each and all of the reviewers of steps 2240 &2250, are sent to notify the primary medical professional thatreviewers' comments are complete. A primary medical professional canthen schedule his time or immediately start his own review as depictedin step 2270.

FIG. 22, step 2270, shows a process step for a primary medicalprofessional to select a subset (one, many, or all) of comments within asuperset of comments produced by self (step 2240), additional medicalprofessionals (step 2240), and automated processes (step 2220), to beincluded in a final report. Once the comment selection has been made,the finalization of the report is completed through step 2280. A finalrecord, albeit printed, electronic, or both is memorialized. A currenttypical procedure would be to print out the report to be filed into apatient record. However, in the last few years, Electronic MedicalRecords (EMR) have become more popular as technology in this area hasexpanded, matured, and become more reliable. Another service opportunityto a physician's office is available at this step, step 2280. Theobjective of embodiments of the present invention is to have the productof a high resolution imaging system and management thereof for agastro-intestinal tract. Part of this system could be a high resolutionprinted report. Current Capsule Endoscopy systems do not provide forresilient, high quality printed images as a result of the low costrequirement on office co-located equipment for the medical professionalstaff. An objective of embodiments of the present invention is thesystem and method for producing a very low initial and residual cost ofproducing very high quality printed images and reports. In step 2280,the selection of a print service is an object of embodiments of thisinvention. A print service would acquire a high volume and high qualityprint equipment, and have access to the final report created from step2270. In an embodiment, this remote, electronic system would trigger anautomated printout, associated with general information about themedical professional and office, such as contact information, address,and optionally express mail account numbers and related information. Instep 2280, office information, an automated print out of a report, plusa shipping document would facilitate minimal human involvement andreduced residual labor charges. An automated printing service withmultiple clients could then produce very cost effectively extremelydurable high quality image reproduction for permanent record storage,transporting this document back to the originator's office. And,finally, in step 2290, preconfigured alerts would be dispatchedindicating that a final report is available. An example of such an alertwould be to an internal office staff for scheduling a patient visit, orpotentially an automated scheduling calendar would generate anelectronic patient visit request sent directly to the patient's email,phone, or similar electronic device. Steps 2240 through 2280 are furtherdiscussed in the next sections.

Image Display System

FIG. 8 depicts a system 800 for displaying a plurality of differentaspects of images captured by an ingestible scanning capsule. Anoperator, 810, interfaces with system 800 and controls position andaspects by interacting with various user interfaces, including but notlimited to joystick 820 and keyboard 821. The interface devices areconnected to a host computer 841. Host computer 841 displays informationand/or an aspect 831 of a current selected position, as it collectsinformation from devices (not shown) attached to network 802. Hostcomputer 841 also directs slave computers 842 and 843 to displaydifferent aspects of the same location of the GI tract currently shownon the host computer. In an embodiment, each of slaves 842 and 843 havetheir own computing processor and directly attached monitor displayingaspects 832 and 833, respectively. In an embodiment, control from thehost to slaves occurs across network 802 through network connections851, 852, and 853. These network connections can be, industry standardconnections, such as, for example, Ethernet, Wireless, or Cellular datanetworking, and other types of connections that will be apparent tothose skilled in the art.

FIG. 9 depicts another embodiment of a display of multiple simultaneousaspects. A computer 941 is suitably selected to have sufficientprocessing power and memory to process and display multiple aspects.Operator 810, interfaces 820, 821, network 802, and network connection851 are the same as shown in FIG. 8. Multiple monitors displayingaspects 831, 832, 833 are all connected to a same computer 241.

FIG. 10 depicts another embodiment of a display of multiple simultaneousaspects. A computer 1041, similar in nature to the computer 941 of FIG.9 has only one display. However, aspects 1031, 1032, 1033 all appear insub-sections of the full monitor. Operator 810, interfaces 820, 821,network 802, and network connection 851 are the same, as depicted inFIG. 8.

FIG. 11 illustrates a display 1100 as is similar to that of FIG. 10having multiple aspects and controls on one display unit. A traditionalendoscope aspect, referred to as looking down the tube is shown in area1130. The dark spot shown as 1133 is a part of the GI tract too distantto be in focus or illuminated. The aspect of area 1130 shows verylimited overall imaging. Traditional endoscopes and even the neweringestible camera devices only provide this limited viewing picture.Diagnosing physicians are not impeded by this aspect with traditionalendoscopes in that they have to take the time to navigate the endoscopeso as not to puncture the GI tract in the patient, as this procedure iseffectively real time with the imaging. However, in newer ingestiblecamera endoscopes this traditional aspect forces the diagnosingphysician to weed through picture by picture almost one at a time to beable to get an overall view of a patient's GI tract. Image datacollected by a scanning imager capsule provides an overall view thatgreatly enhances the use of a viewer's time. This aspect, is shown inarea 1110, and depicts virtually the result of removal of the GI tract,stretching it into a straight line, cutting it open on that straightline, and folding it open. Since the GI tract of a human, for example,is very long with respect to its circumference, or width as would befolded open, the aspect shown in area 1110 shows multiple sections. Forexample, images of the mouth and throat would be on the left side of thetop section. The right side of the top section and the left side of themiddle section might show the beginning of the small bowel. The imagethrough the small bowels may continue throughout the middle section andinto the start of the lower section. The image data just prior toexcretion would be located to the right of the last section. A personskilled in the art would realize that any number of sections could beused without departing from the spirit and scope of this invention.

Since an entire GI tract displayed, as is in area 1110, is very smallcompared to potential issue needing further investigation, it isconceived that only summary (low resolution) information would bedisplayed in area 1110. Judging from the summary images in area 1110, aspecialist may want to zoom in to a higher resolution image. An area1120 provides a close up image of an area of interest. This close uparea is also displayed for reference as subsection 1111 of area 1110. Aclose up area 1120 has a known position with respect to the entire GItract. Statistical data corresponding to this is shown in area 1160.Data that can be shown is total time from entry to exit of the GI tract,and similarly, a relative time from entry as well as a relative time toexit is also displayed. More useful, however, is distances. Since theingestible scanning capsule can be located with precision an entirelength of the GI tract, the distance from entry, and the distance toexit can be displayed for use by the diagnosing physician. Distances areextremely useful in determining which of other procedures would beappropriate. For example, if an area of interest were 20 inches from themouth, a traditional endoscopy procedure starting from the mouth wouldbe appropriate while a colonoscopy starting from the rectum would not bean appropriate procedure. Similarly, an area of interest 6 inches fromthe rectum would suggest the use of a colonoscopy over the endoscopyprocedure that starts from the mouth. Since the area 1110 is a displaybased upon distance, a time display in area 1160 could indicate areaswhere peristaltic action is faster than or slower than some criteriaindicating problems that need further investigation by the diagnosingphysician.

A specific anomaly of the GI tract is depicted in the multiple aspects1110, 1120, 1130 and in the location of the GI tract currently selectedby the operator is shown with 1112, 1122, and 1132, respectively. Anexample of an anomaly would be a spot of blood. A person skilled in theart would recognize images of a multitude of anomalies. All anomaliescan be shown on area 1110 as it depicts the entire travel through the GItract. One such anomaly depicted is anomaly 1113. Anomaly 1113 is acolor enhancement of a non-visual sensor and/or imager. Examples ofanomaly 1113 would be high intensity reflections of UV, as is known forcancerous growths, high intensity reflections of IR indicative of a highvolume of blood flow, detection of a concentration of heat indicative ofcellular activity and/or infection, non-uniform reflectivity ofultrasonic waves in part indicative of dense, absorbing tissues such aspolyps and other potential growths not found in typical tissues, andother such sensors. Controls that tarn on and off color enhancements aswell as controls setting thresholds for sensors or imagers not in thevisible light spectrum (not shown) would give an operator of the presentinvention a method to easily highlight for further evaluation (as inzoom in) certain areas of the GI tract. This process greatly eliminatesthe time necessary to accurately detect areas of concern within the GItract of a patient.

An operator, 810, once determining an area of interest is allowed toannotate their findings in area 1150. The location within the GI tract,hence within the scanned data in combination with the ability tohighlight, circle, or otherwise indicate a zoomed in area (such asstoring the configuration of area 1120) is also stored, as an area ofconcern or finding as a distance into the GI tract, a radian angle and azoom aspect. Additionally, the operator information and optionallyqualifications (or index thereto) are also stored with the annotation ofthe area of concern. A person skilled in the art would realize adatabase of annotations with index into the scanned data at a certainpoint and operator database index could be an efficient mechanism tokeep a record of the areas of concern for each individual use of aningestible scanner. Once such an area of concern database is built, itcan be very functionally utilized to skip around very quickly to justthose areas that should be studied more closely or even reviewed withthe patient. In an embodiment of the present invention, a report ofselected or all areas of concern can be created from the annotationsstored at the time of review, or at a later time or different computingplatform remote or otherwise.

An embodiment of this present invention affords an opportunity forconstruction of an expert software system or autonomous graphicalpattern recognition system to parse through a patient's scanned data andautomatically generate areas of concern. On FIG. 11, such an expertsystem is demonstrated in area 1140, for example a “UV Reflective”category has been selected by operator 810. Such a selection is a filterto accept any area of concern information from an expert system, in thiscase, that recognizes certain scanned data from a UV emitted lightsource as being above a certain level of reflectivity and concentrationthat would potentially indicate a cancerous growth. In this invention,operator 810 has the ability to select which filters he or she isinterested in observing for a particular patient. Once a selection ormultiple selections are made, the operator can utilize controls 1145 toskip forward or backward through the index of areas of concern. It isimportant to recognize that both expert systems and also specific humanreviewing authorities (diagnosing physicians, specialists, etc) aregiven as filter criteria allowing an operator to select specifichistorically accurate resources for a patient's current health issue orconcern. In the aspect shown in FIG. 11, area 1110, for example, thereare two areas of concern selected by filters for UV and Dr. Peters. Thecurrent area of concern shown relates to Dr. Peters and is alsoindicated in area 1150 showing the diagnosing physician's annotation ofthe area previously reviewed. The operator can select a next area ofconcern skip function by clicking controls 1145. The next area ofconcern, as indicated by 1113 would then be displayed with differentaspects in areas 1120 and 1130. Alternatively, the controls 1145 may belocated or duplicated as specific buttons on joystick 820, or specifickeys on keyboard 821, and so on as would be anticipated by a personskilled in the art. There are two areas of concern demonstrated in area1110 but are not included in the skip function as the resource was notselected in area 1140, as indicated by several sources not having aselection icon matching the name of the source.

An operator, such as person 810 in FIG. 8, is able to navigate theaspects of areas 1110, 1120, and 1130 in a number of ways. A linearmovement, as would be the case with pushing and pulling a standardendoscope (i.e. up and down the tube), is controlled with either thejoystick 820 of FIG. 8, the keyboard 821 of FIG. 8, or a mouse click oncontrol icons 1135 of area 1130. Additionally, the operator may usecontrols 1145 or a duplicate set connected with buttons on joystick 820,to skip directly to areas of concern. It is also anticipated by thispresent invention to afford controls for zoom and rotation for areas1120 and 1130, such as might be a left and right twist of a joystick anda straight left and right joystick manipulation for zoom and rotation,respectively.

An ingestible scanning capsule stores very specific location data witheach scanned image data. Location data is computed with reference to afixed point, typically reference to a location of a belly button of ahuman subject as a permanent point of reference. An objective of thepresent invention is to be able to display a path of travel over theduration of time an ingestible device passes through a GI tract.

FIG. 12 depicts a normalized graphic of a typical human subject 1210,with an exemplary GI tract 1220, and an exemplary travel path 1230 of anobject that might pass through the human subject. FIG. 12 is forreference as a point of demonstration of the present invention.

In an embodiment of the present invention, an additional aspect ofdisplay for display 1100 (or additional monitors as depicted in FIGS. 8,9, and 10) is explained with reference to FIG. 13.

FIG. 13 schematically shows a normalized depiction of a human 1310 for abackground reference. A total traveled tract 1320 is computed andoverlaid upon the background. A current location or area of concern ischosen for display on multiple aspects and is graphically represented bya control and icon 1330, control 1355 and control 1365. Control 1355depicts a linear progression in distance from mouth (or first acquiredlocation) to anus (or last acquired location). Control 1365 depicts alinear progression in time from ingestion (top) to excretion (bottom).Software allows movement of the controls 1355 and 1365 only up anddown—for example by clicking a mouse and dragging the control up ordown. Control 1330 is more complicated, in that 3 directional displayand movement can be accomplished. In an embodiment of the presentinvention, a mouse drag on a 2 dimensional (x, y) level can move thecontrol/icon 1330 around the background 1310. When the control isreleased (for example release of the mouse button), the control/icon1330 is placed upon the closest point of the path 1320. To achieve a 3dimensional control, a rotation control will spin the entirety of 1310,1320, and 1330 as it was on an axis from top to bottom and in the centerof the body.

An additional spin control could optionally be added to the aspect ofFIG. 13. Alternatively, a joystick 820 twist control could accomplishthe same spin function. In an alternate embodiment, areas of concern canbe denoted by icons placed upon the travel path 1320 as would beconsidered by persons skilled in the art. An operator may quicklydetermine what additional procedures may be needed by a quick glance ata distance indicated by control 1355, which graphically andsimultaneously depicts distance from entry to area of concern and alsodistance from excretion to area of concern. A diagnosing physician couldthen easily tell if an endoscopy or colonoscopy follow-on procedurewould be appropriate.

FIG. 14 schematically shows the functionality of display of a path, andarea of concern directly upon a patient. An area of concern may generatea request for additional procedures such as ultrasound analysis, MRIimaging, X-ray imaging, or even exploratory and other types of surgicalprocedures. All of the above can benefit from time savings, costsavings, and reduced impact upon the patients impact from a surgicalincision size and/or prep area. A projection of path 1320 and areas ofconcern 1330 from FIG. 13 is demonstrated as path 1420 and areas ofconcern 1430, respectively in FIG. 14. The projection device may be assimple as a typical office projector as would be typically be utilizedfor meeting presentations. Other devices could be utilized withoutdeparting from the spirit and scope of this present invention. Asdescribed earlier, a fixed point of reference, for example belly button1415 is stored with the location information of the path 1420. A firstprocedure would be to align the projection so as to fit the indicationof belly button 1415 with the patient's actual belly button. A secondprocedure would include any necessary rotation to align entry (mouth)and exit (anus) of the tract 1420 projection so as to align to thepatient 1410 without rotation off axis. A third procedure would be tosize the display to fit the proportions of the patient 1410. A zoom in &out control (not shown) or a physical movement of the projection devicetowards and away from the patient will be performed until the entrypoint 1440 and the excretion point 1450 of the tract 1420 match thepatient's physical location of mouth and anus. Referencing FIG. 11, askip from area of concern, to a next or previous area of concern willilluminate an area of patient with icon 1430. The icon may be utilizeddirectly to capture data from another procedure. Additionally, the icon1430 may be utilized indirectly by further marking upon the patient theindex or other indication of the area of concern for procedures to takeplace at a later time or in another facility or room.

FIG. 15 is a flow chart of a main processing system of image processingsoftware for rendering the multiple aspects and views as depicted inFIG. 11, according to an embodiment of the present invention. By way ofexample only, FIG. 15 flowchart shows a loop function. As is typicalwith a loop function in a display scenario is an initialization phase asshown with steps 1502-1504, a graphical drawing function or functionsshown with steps 1510-1530, a control or input and processing of theinput as shown with steps 1540 through 1571, and loop control returnedback to graphical updates in step 1530.

At step 1502 the main processing routine is initialized specifying inpart a first position, colorization, and magnification level, the firstposition at the start of a full image available from a series of singlescans, the result of processes shown in FIG. 22. At step 1504 allfilters are turned off Filters can be selected by an operator of thesoftware to include or exclude numerous comments on abnormalities foundby the person or process, or groups thereof, that have placed commentsupon the full image of a particular gastro-intestinal scan of a patient.For example, pre-selected automated analyses processes for celiacdetection may afford a reviewer an opportunity to initially quicklyselect the automated processes, or more specifically a trusted celiacfinder automated process and skip (see FIG. 11, item 1145) directly toan area of interest with a suspected abnormality such as celiac disease.Similarly, in another example within embodiments of the presentinvention, a pre-selected trusted reviewing, entity, employing humanlabor potentially in a low labor rate country such as India, may havepre-viewed the scanned image, made comments on certain portions of theimage, and completed their independent analysis for a primaryreviewer/doctor to initially and quickly select their findings and jumpto the view indicating, for example, an area of discolorizationabnormality indicating a suspected irritable bowel disease.

FIG. 15, steps 1510 and 1520 initialize a graphic display or backgroundand are explained further in a following section. The initializations of1510 and 1520 will provide a display for substantially non changingportions of materials on the displays as a reviewer starts to view acapsule endoscopy scanned image. Step 1530 is also defined in detail infollowing sections and FIGS. Step 1530 updates non static graphicalinformation, for example areas 1120 and 1130 in FIG. 11, and also whenmoving through a magnified image from an overall single scanned image ofa patient's gastro-intestinal tract. After the system and displays havebeen initialized, user input for navigation and commentary is acceptedas is depicted in step 1540. Depending upon the type of input, asdetermined with steps 1541 to 1549, algorithms are deployed in steps1561 through 1571, and loop control is returned to update the display(s)in step 1530 as would be appropriate. Step 1541 affords a reviewer anability to select and deselect a variety of comments to review fromother reviewers and/or automated processes. If a selection is made, acorresponding action 1561 toggles a specific filter on or off asrequested by the reviewer.

Once the selection is made, control is passed back to the input loop instep 1540. Step 1542 affords a reviewer its own opportunity to save acurrent viewing magnification level, colorization, and position alongwith a comment about a specific area and a potential for an abnormality.Step 1562 saves in a multitude of indexes and databases the informationof the comment as well as the other information about the current view,location, magnification level, colorization, etc. It is important tonote that, while the current FIG. 15 demonstrates a textual comment, itis by example only Embodiments of the current invention also anticipatethe use of video commentary, audio commentary, and also chalk talk styleoverwriting of shapes and freestyle drawing (of a variety of colors) forthe commentary of a particular graphical image. Examples of each are notincluded for conservation of text length, but are anticipated by thisinvention.

Step 1563, 1564 affords the reviewer in a current magnification level(zoom level) to scroll forward and reverse, respectively, with respectto the fall gastro-intestinal image and, for example, forward being tothe right, and reverse being toward the left. Additionally, a movementmay be defined as being a percentage of the current magnified view, suchas 50% (half of the image remains on the magnified view as a new imageis displayed). Embodiments of this invention anticipate several methodsof image movement—i) a more simple form of movement which is to re-drawa new image, ii) move a portion of an image on the display withoutreference to a full image file while accessing that file for only a newportion of the overall magnified image to display, and iii) a morevisually soothing smooth scroll of an image by continually redrawing theimage only a few percent of change, for example 25 times redrawing animage moved only 2% in a certain direction. The latter smooth method,although more technically difficult, is certainly more soothing to thehuman eye and brain, and likely worth more to a typical reviewer of amultitude of images frequently. The program updates the new locationcenter, magnification level, colorization, etc as appropriate for userinput, then passes loop control back to step 1530 to perform the updateof displayed information from a new aspect, and then on to step 1540 fornew user input.

Step 1545, 1546 accepts input from a reviewer to launch a timed stepforward and reverse, respectively, without the need for additionalconsecutive inputs by a reviewer. Effectively, then, steps 1545 and 1546advance a magnified image forward and reverse such as with steps 1543and 1544 respectively and further updated images from steps 1563, 1564,respectively, and then back to 1530. This advance is a repetitiveadvance initiated by a single user input and continued until either, i)the end or beginning of the full image (respectively) is reached, oruntil an additional input from a reviewer is received. The additionalinput of this example might be a second input, such as is with a toggle,of the initial input, such as a first FFwd initiates a forward continualmovement, and a second FFwd terminates the forward continual movement.In an alternate embodiment, not shown but inclusive in this invention, afirst FFwd would initiate a slow, continual movement forward, a secondFFwd would initiate a medium speed movement forward, and a third FFwdwould initiate a fast speed movement forward through the full image. Anobject of this alternate embodiment would be a termination of themovement forward by either a user input of a PAUSE function (not shown),and/or a selection of any other movement inputs such as the forward,reverse, FRev, Comment, Filter Selection, and so on. Still further, atypical keyboard (FIGS. 8 & 9, item 821) ‘Esc’ or <space> or othertypical keys are an anticipated movement termination user inputs. Steps1545 and 1546 call upon the functions 1565 and 1566, respectively, toupdate a location, continually and autonomously, while returning loopcontrol back to update of the display image(s) in step 1530 andadditional user inputs in step 1540.

Step 1547 affords a reviewer of an image and its corresponding,magnification level a direct jump to a specific location within the fullimage. An example of a typical implementation would be a mouse click onthe full image, while this input would display an additional magnifiedview of this location upon another aspect. The location is selected,optionally selecting a default magnification level, as a part of process1567, and then returning, loop control back to updating displays withstep 1530.

Steps 1548 and 1549 correspond to a reviewer's request to jump to apre-defined location, magnification level, and colorization as storedand indicated in a previously commented step. Furthermore, thesequential stepping through of comments are available, but only forthose comments by reviewers and/or automated processes that havespecifically, or by group, selected through the filter selections ofsteps 1541 and 1561 by the current reviewer. It is anticipated that anembodiment of this invention would allow a configuration, not part ofthis specific procedure FIG. 15, to pre-select a certain common set offilters, customizable to any specific reviewer as part of a loginprocedure. Since these comments in this embodiment are considered to bea list which secondarily then can index a location within a specificfull image, two steps may be performed, such as 1568 which selects anext comment from a list of comments, and subsequently step 1570 whichselects and sets a specific location (magnification, and colorization)from attributes of a present comment prior to returning loop control tosteps 1530, updating the displays, and 1540 for getting a next userinput.

FIG. 16 is a flow chart of a subroutine of the image processing softwarefor rendering a linear aspect of the displays depicted in FIG. 11,according to an embodiment of the present invention. FIG. 16 is also amore detailed expansion of the step 1510 of FIG. 15. FIG. 16 is providedas an example of an optional routine to render a single image (withoutrespect to scrolling within a window) from multiple individual pieces ofthat image, called frames, from a storage unit rendered upon requestreal time or pseudo-real time from a reviewer of the full image.Optionally, FIG. 16 could be implemented upon image acquisition or batchmode during, an overall image acquisition process and stored as a fullimage for later real-time viewing. However, FIG. 16 will overallappropriately fit a full image, possibly scrolled, into an availabledisplay space commonly referred as a ‘window’. FIG. 16 is one example ofa method to display a full image into an available display space. Otherimplementations do not depart from the spirit and scope of this presentinvention.

Step 1610 of FIG. 16 demonstrates an initial opening of an image baseddata file containing a multitude of scanned images comprising in total afull scanned image of a gastro-intestinal tract. It is implied thatthese individual frames may, but may not contain duplicative informationbetween adjacent frames. Step 1612 calculates, from information withinthe image database, a total quantity of frames contained within the fullavailable image. Step 1616 calculates the number of available pixels inthe current available display space from, for example, interrogation ofthe operating system under which this procedure is currently miming.Step 1618 calculates from the preceding information a number of framesthat will need to be compressed into a single row or column of pixelsbased on the uniform distribution of image data across a non equalnumber of pixels available to display the image within. Step 1620 is aninitialization of a loop to then render the fall image across anavailable display.

Step 1630 initiates a loop of sections. A full contiguous image of agastro-intestinal tract is assumed to be long with respect to the width,if each pixel of data is square—or non-distorted in an x and ydirection. Effectively, for demonstration, if a gastro-intestinal tractwould be removed from a patient, cut and splayed open so as to provide aflat surface, it would be approximately 300 inches long while only 3inches wide. In order to fit this object into a more square displayarea, it would be ideal to evenly divide these 900 square inches into asay, 30 inch by 30 inch area. More specifically, a full 300 by 3 inchimage should be sectionalized into 10 of 30 horizontal by 3 verticalinch sections, that would be then horizontally stacked one upon theother. While this is an ideal sectionalization for example, it isanticipated by embodiments of this invention to also have spacing,rulers, and other text and graphics to be applied within the overalldisplay window for best viewing as would be determined by an expert ingraphical user interfaces (GUIs). For purpose of simplicity, thesedetails are intentionally left out of the example, but are impliedwithin the spirit and scope of this present invention.

So, step 1630 has calculated an appropriate number of sections todisplay, and initially sets up for display of a first of potentiallymany sections. Step 1640 initializes a loop controlling where in thegraphical display a current rendering of a row of image data should bedrawn. Step 1650 then moves through the image data, step 1652 collectsinto the program memory that image data, step 1654 compresses or expandsas necessary frame pixels to display pixels, for example averaging amultitude of image data pixels to substantially less display pixels.Step 1656 simply loops back to step 1650 to get all frames available fora particular column for display. Step 1649 as shown both displays(renders) the results of 1650 through 1656 onto the display/window, thenadvances to the next column, p, of the display by looping back to step1640. Step 1639 similarly advances to a next section to display in thewindow, and returns loop control back to step 1630 for a next section.When all sections have been rendered onto the display, program exitsfrom step 1639.

FIG. 17 is a flow chart of a subroutine of image processing software forrendering a drawing of the full GI tract in 2D as is shown in FIG. 13,according to an embodiment of the present invention. This embodiment ofthe present invention assumes a reasonable accuracy of location withinthe image acquisition system. One embodiment of such a system would bean acoustic data transmission system. Such a system with accuratelocation ability is defined in publication WO2008-030481-A2,incorporated by reference herein in its entirety. In such a system forperiodically gathering image data, as a result also location informationis stored within the data. Embodiments of the present invention refer tothis data storage as a frame database. However, a storage of locationinformation with image data based upon a real human subject is notnecessarily nor frequently of the same dimensions as an output display.Hence, a translation, or resizing of the location, and potentiallynormalization and stabilization of the location information, may benecessary and an object of this routine shown in FIG. 17.

Step 1710 opens an associated image file, or frame database. A first andlast entry of location information, for example, would indicate entry(mouth) and exit (anus) locations of the body as would be the case witha location based upon acoustic location systems. However, additionalinformation, such as signal attenuation, would be likely used in an RFbased location system, as an example of an alternate embodiment of astorage of location within the body of a specific image acquisition.Step 1712, nonetheless, finds a entry and exit location information.Step 1714 renders a standard picture, a generic drawing, or optionallyan actual picture of a human subject. Step 1716 stores a defined maximumspace available for the display. Step 1718, then, with size informationfrom an available display and an available location parameters from agastro-intestinal transit, computes a ratio so to fit the locationinformation from the transit onto the static painted image on thedisplay. Starting back from the top of the image data base with a firstlocation, a loop is setup with step 1720 (first record start), and 1722(a computation of an absolute pixel location from a relative base oflocations stored within or as an index to the image database. Step 1730depicts a start of a loop that reads locations with times of signaldetection (relative to ingestion and/or absolute time reference). Step1731 gets a database entry and related records in other associateddatabases, for example location and time references. Step 1732 stores,in 3 dimensional representation, a path from a previous database entryto this current database entry and in addition accumulates a distancetraveled as a path length, or a conversion from 3 dimensional travel toa 1 dimensional top to bottom reference. Step 1733 stores a relativetime offset from a first and last time available for this path. Step1734 applies a ratio to this path, and applies this in step 1735 whenrendering a path (line) onto the display/window. When all records havebeen evaluated, loop exits at step 1739. Step 1740 draws a distance barbackground graphic with the accumulated total distance traveled fromstep 1732. Step 1750 textually depicts on the display/window thebeginning and final points on the background bar from step 1740. Thedistance bar graphic and textual information is as seen in FIG. 13,items 1350 & 1355. Similarly, step 1760 and 1770 draw another graphicbar and corresponding textual references, this time on the basis oftime, as opposed to distances of 1740 & 1750. Steps 1760 & 1770correspond to items 1360 & 1365 on FIG. 13.

FIG. 18 is a flow chart of subroutines of image processing software forupdating multiple aspects of the displays and providing annotations asdepicted in FIG. 11, according to an embodiment of the presentinvention. Step 1530 of FIG. 15 is a general title for a set of routinesfor a variety of aspects of a selected position within a full image of agastro-intestinal tract made by, for example, an embodiment of ascanning capsule endoscopy product. Step 1530 includes a multitude ofsubroutines for a multitude of aspects. These subroutines are intendedto be able to reside within one processing environment and also be ableto be split into multiple processors. Additional aspects of a fulldisplay system are likely. FIG. 18 demonstrates by example only.Additional aspects or removal of aspects from the details of FIG. 18does not depart from the spirit and scope of this present invention.

FIG. 19 illustrates flow charts of subroutines of the image processingsoftware for updating a tract aspect and updating time and distance,respectively, of the displays depicted in FIG. 11, according toembodiments of the present invention. FIG. 19 is a further detail ofstep 1830 and step 1850 of FIG. 18. Step 1830, as described in FIG. 18,updates a tract display of FIG. 13. As shown, FIG. 13 is a 2 dimensionalrepresentation of a 3 dimensional tract taken by a scanning capsuleendoscope, for example, over the course of an exemplary traversal of thecapsule over time while in communication with an external receiver. As ahuman operator of embodiments of the present invention moves around animage of the gastro-intestinal tract by a variety of methods, the tractaspect will need to show a corresponding updated display of the currentposition of the current selected position. Step 1910 then is thebeginning of the procedure to update the tract display. Firstly aprocedure removes old, non valid information such as thecursor/crosshair, such as depicted as item 1330 on FIG. 13. Step 1912re-computes a new position of the cursor. A desired position within thefull GI tract is available from the system, and a correlated positionwith respect to a current ratio for display of the GI tract, is thencomputed. This computational result is used in step 1914 to redraw acurrent position indicative cursor on a 2D representation of a 3D GItract. Similarly, step 1916, 1918 and 1920 removes, recalculates, andredraws, respectively, a indicator of position within the full length ofthe GI tract as is represented by item 1355 in FIG. 13. Additionally,and similarly, steps 1922, 1924, and 1926 will remove, recomputed, andredraw, respectively, an indicator of time elapsed from swallowing acapsule to reach a current position that is in view in a magnified view.Steps 1910 through 1926 offer by way of example only an update to threecursors/indicators. A person skilled in the arts may elect to have onlyone, two, all, or supplement information with additional informationrelative to information stored in the position and full imagedatabase(s) without departing from the spirit and scope of this presentinvention.

FIG. 19 additionally demonstrates a detailed procedure 1850 to update atime and distance display associated with a current magnification viewof a full GI tract as exemplified by FIG. 11, items 1120 and 1110respectively. Steps 1950 utilizes a user requested position from inputwhile viewing a primary aspect, and calculates a distance from astarting point for a full length of a gastro-intestinal tract as isdisplayed on item 1110. The distance computed may be represented in anumber of units, for example, inches, centimeters, etc, but wouldindicate a real position of the capsule that captured the image datacurrently viewed, as opposed to a distance on screen. Additionally, inan alternate embodiment, a scale of units would appear next to either afall GI tract (item 1110), or a magnified portion of that tract (item1120), or both so that visual objects could be visually sized quicklyand without much error on the part of a human operator. In thisalternate embodiment, a user requested change of a magnification levelwould also instill a change, and re-drawing of the previously discussedscale upon the magnified image 1120.

Furthermore, in another embodiment, a distance from a starting point maybe selected as a first location (mouth), but it is also anticipated thata first part, for example image data from mouth to first part of smallbowel, and a last part, for example image data of a colon and anus, ofthe full image, and operations within may be limited to a designatedfirst and last position as a subset within the full image, to compute astarting position, a current position, and an ending position asrepresented in item 1160 of FIG. 11. Returning to FIG. 19, steps 1950,1952, 1954, 1958, 1960, and 1962 calculate real distances that aretraveled by a scanning capsule endoscope with respect to a start and endposition that can be either a first and last position entry, oroptionally defined as a subset, such as a first image data and lastimage data of a small bowel, a small and large bowel, etc. The optionaldefinition may be selected by a human operator or also an automatedprocess through image or colorization recognition algorithm, or acombination of the two. Step 1952 computes a real traveled distance froma current magnified view as selected by a user to the end position asjust defined. Step 1954 formats the information of starting distance,distance of current position from start, distance of current position toend, and total end position, or length of capsule traversal. Step 1956displays the formatted position in an aspect and as depicted as item1160 of FIG. 11. Additionally, steps 1958, 1960, and 1962 compute andformat times of transit with respect to the corresponding distances of1950, 1952, and 1954, respectively. Step 1964 displays updated time oftransit information in an aspect depicted as item 1160 of FIG. 11.

FIG. 20 is a flow chart of a subroutine of image processing software forupdating a zoom aspect, according to an embodiment of the presentinvention. A full gastro intestinal tract display of FIG. 11 is shown asitem 1110. As a user of embodiments of this present invention navigatesaround this full image, a window of a current magnified version is shownas item 1111 as a subset of item 1110. Item 1111 is shown to be a subsetwidth, but a full height and is for example and simplicity only. It isanticipated and an embodiment of this present invention that a subsetheight and width for item 1111 is achievable by a reviewer'smanipulation of inputs to the system. A subset image, then, can beenlarged and is shown in FIG. 11 as item 1120. As demonstrated, anabnormality of item 1112 in the full image is also enlarged and shown asitem 1122 at the same magnification level as requested by a reviewer'sinput. The item 1120 is updated, or re-drawn through the procedure ofFIG. 20. Step 2010 and 2012 retrieve two important variables and are aresult of reviewer input—a current magnification (zoom) level, and acurrent location with respect to a full gastro intestinal image,respectively. User input for control of these two variables isanticipated to be similar to a well known user interface—a Google mapssystem. For example, to increase a magnification level, a user couldscroll a mouse wheel. An additional example, a user could click on aposition within item 1110 and skip directly to area at a currentlystored magnification level. However, as an additional method of imagemanipulation that is not typical with a Google map type of application,is a user request to move to a previously commented material stored inassociation with a current full gastro intestinal image. A comment couldbe selected from, for example, a mouse click on a line of text in item1150 of FIG. 11. Alternatively, a page up or down key could also, forexample, move through previously commented material. Selection of acomment would move a zoom indicator, item 1111, and a zoom display, item1120 to a specific location, a specific magnification level (stored withthe comment), and other previously stored configurations such ascolorization, brightness, color enhancements, etc. Although theseadditional stored configurations are not shown on FIG. 20 forsimplicity, use of additional stored configurations of a graphical imagedoes not depart from the spirit and scope of this present invention.

Steps 2014 and 2016 gather and calculate information about the size ofthe display area. Step 2018 and 2020 then calculate a ratio of how manypixels in an image database to translate onto a certain number ofdisplay pixels. As would be anticipated, a compression or an expansionon a pixel basis could occur, depending upon the display size for themagnified version, and the zoom, or magnification level requested by anoperator of the system. Step 2022 opens an image database, and isoptional. In some configurations, a software may already have the imagedatabase opened. In alternate embodiments where a zoom aspect isoperating on a separate hardware system, the image database may need tobe opened, or reopened upon an update of location and magnificationlevel. Step 2030 begins a loop of rendering a line of graphicalinformation, which is rendered after gathering all information in step2039, also step 2039 increments to a next line until all lines requiredto update the display have been rendered. Step 2040 begins a loop ofimage data accumulation within the outer line loop of steps 2030 & 2039.Steps 2040, 2042, and 2049 acquires and manipulates image data from adatabase, a subset at a time—for example a frame of image data at atime. On, for example, a low magnification level request, many frames ofan image database may be averaged together to form a single line of adisplay of the data. Alternatively, a large display may require thesoftware to expand a single frame of data onto multiple lines of thedisplay. So, step 2040 sets up a loop for multiple image data framesfrom a database, step 2042 retrieves the data from the database for aparticular frame, and step 2049 compresses or expands the data asappropriate. One embodiment would compress image data by way ofmathematical averaging of multiple pixels of information into onerepresentative pixel. An alternate embodiment may deploy more advancedalgorithms rather than a simple mathematical average. Such advancedalgorithms are well know to those skilled in the art and their use doesnot depart from the spirit and scope of this present invention. Step2049, in addition, will increment to a next, image data frame, ifavailable and if necessary.

FIG. 21 is a flow chart of subroutines of image processing software forupdating a tube aspect, according to an embodiment of the presentinvention. The tube aspect is depicted in FIG. 11 as item 1130. Acomplete, single, and full image of a gastro intestinal tract as in item1110 is displayed as would be imaged in a topographical view. Thecurrent products in capsule endoscopy provide a typical endoscopy viewto a reviewer, which is a camera view point in a gastro-intestinaltract, or tube, pointed and focused in an axial direction. Embodimentsof the scanning capsule endoscope described herein do not offer directlythis camera view point. Thus, to be compatible with an existing reviewerexpectation, or backward compatible for viewing comfort, it may benecessary to provide this tube aspect. Since the image does not directlyexist as a result of an imager output, it must be created. FIG. 21demonstrates a simplistic embodiment of an algorithm that will render anaxial view from an otherwise non-axially collected image data. Numerousalgorithms are in place to render a different viewing angle from anotherwise initial set of image data, mostly in military applications.For example, public war time satellite imagery of ground terrain hasbeen demonstrated to be viewed from a viewpoint of a virtual airplaneflying close to the terrain through software data manipulation. Lessviewed but still publicly demonstrated of recent years is in the worldof sports. During the Olympics, several cameras tracked snow boardersgoing down terrain, and a virtual 3-D viewing angle capability wasdemonstrated and again in software manipulated image data to extrapolatefrom multiple actual images what an image would appear like if viewedfrom an angle that was virtual. A similar, but not identical processcould be applied to a scanning capsule image data to render a scanningdata collected on a pill surface as if it were viewed by a cameralooking in an axial direction.

Firstly, in step 2110 and 2112 a procedure collects the center and size,respectively, of an area for display, such as in FIG. 11 item 1130. Step2116 opens a scanned image database containing frames of image data.Step 2118 collects a current location as directed by a human operator,or reviewer, of embodiments this invention. Step 2120 creates an imagearray subset from a full image available in the database. The imagearray will have a starting point at the location indicated by the humanoperator, and an ending point at a defined point either forward orreverse into the database, the direction indicated by a direction ofmovement as directed by the human operator. Step 2130 through 2139 thenmathematically manipulates the 2D scanned data subset array into anotherarray that is organized circularly and subsequently renders the circularimage data onto the tube aspect display window. Step 2130 initiates aloop of circles from outside, to inside, for example. Step 2132retrieves scanned, linear image data and then in step 2134mathematically fits a linear row to a circle of a single pixel depth,for example curving a line of image around a circle of a certain radius.Step 2136 stores the circular image information computed from step 2134.The process continues back to step 2130 for successive linear rows ofinformation.

All rows of scanned image data have the same row size in pixels.However, in a circular representation, the number of pixels in a row ofan outside radius is more than a number of pixels in an inside radius.Therefore, in step 2134, it is implied that the translation will becompressing linear rows of information into lesser pixels in order to beable to fit all of the information into a lesser radius circle. Steps2130 through 2139 then convert an x,y image data presentation into arow, theta image presentation. FIG. 21 is a simple embodiment of theinvention. A person skilled in the art would recognize that depthinformation, feature size, etc can be derived from the scanned imagedata to be able to render 3 dimensional information, such as a height ofan abnormality, such as a polyp, for example from scanned informationsuch as shading and linear depth of shade. This alternate embodiment,although more difficult to produce and illustrate on drawings, is bothanticipated and does not depart from the spirit and scope of thispresent invention.

FIG. 22 is a flow chart of an overall scanned image collection,processing, and reporting system, according to an embodiment of thepresent invention.

FIG. 23 is a detailed flow chart of a scanned image creation process,according to an embodiment of the present invention.

FIG. 24 is an exemplary depiction of scanned data corresponding to theFIG. 23 process.

In an embodiment of the present invention, a report of selected or allareas of concern can be created from the annotations

Illumination

Although the embodiments described above have been described withrespect to some sort of light illumination, the GI tract can beilluminated by various types of sources including white light,multi-spectrum light, narrow spectrum light, infra-red, ultra-violet,and even non-light energies such as, for example, acoustical energy,etc. The type of sensors used to image a scan can be varied asappropriate based on the illumination/sensor combination desired.

CONCLUSION

The present invention has been described above with the aid offunctional building blocks illustrating the implementation of specifiedfunctions and relationships thereof. The boundaries of these functionalbuilding blocks have been arbitrarily defined herein for the convenienceof the description. Alternate boundaries can be defined so long as thespecified functions and relationships thereof are appropriatelyperformed.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the invention that others can, by applyingknowledge within the skill of the art, readily modify and/or adapt forvarious applications such specific embodiments, without undueexperimentation, without departing from the general concept of thepresent invention. Therefore, such adaptations and modifications areintended to be within the meaning and range of equivalents of thedisclosed embodiments, based on the teaching and guidance presentedherein. It is to be understood that the phraseology or terminologyherein is for the purpose of description and not of limitation, suchthat the terminology or phraseology of the present specification is tobe interpreted by the skilled artisan in light of the teachings andguidance.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. It will be apparent to persons skilledin the relevant art that various changes in form and detail can be madetherein without departing from the spirit and scope of the invention.Thus, the breadth and scope of the present invention should not belimited by any of the above-described exemplary embodiments, but shouldbe defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. An apparatus for displaying, a representation ofa gastro intestinal (GI) tract of an animal, the apparatus comprising: amemory that stores a plurality of annular images corresponding to aplurality of portions of the GI tract as an ingestible imaging pillpasses through the GI tract; a processor configured to convert theplurality of annular mages to a plurality of rectilinear images and tomerge a first image from among the plurality of annular images and asecond image from among the plurality of annular images to provide acomposite image, wherein the processor is further configured to applythe merge process only to a first region of the first image thatoverlaps a second region of the second image to average out differencesbetween the first and the second regions.
 2. The apparatus of claim 1,wherein each rectilinear image from among the plurality of rectilinearimages forms a substantially linear projection of the GI tract.
 3. Theapparatus of claim 2, wherein the processor is further configured todynamically display the substantially linear projection of the GI tractin a manner dictated by a human operator.
 4. The apparatus of claim 1,wherein the processor is further configured to permit a human operatorto input a comment related to a portion of the GI tract in the compositeimage.
 5. The apparatus of claim 1, wherein the processor is furtherconfigured to cause graphical user interface (GUI) controls to hedisplayed that are manipulated by a human operator to change a displayof the composite image so as to focus on a particular feature of the GItract.
 6. The apparatus of claim 1, wherein the plurality of rectilinearimages comprises a plurality of trapezoidal images, and wherein theprocessor is further configured to convert the plurality of trapezoidalimages to a plurality of rectangular images.
 7. The apparatus of claim1, wherein the memory further stores specific location information withrespect to a reference point for the plurality of annular imagescorresponding to the plurality of portions of the GI tract.
 8. A methodfor displaying a representation of a gastro intestinal (GI) tract of ananimal, the method comprising: storing, by an image display system, aplurality of annular images corresponding to a plurality of portions ofthe GI tract as an ingestible imaging pill passes through the GI tract;converting the plurality of annular images to a plurality of rectilinearimages; merging, by the image display system, a first image from amongthe plurality of rectilinear images and a second image from among theplurality of rectilinear images to provide a composite image, themerging comprising: applying the merge process only to a first region ofthe first image that overlaps a second region of the second image toaverage out differences between the first and the second regions.
 9. Themethod of claim 8, wherein each rectilinear image from among theplurality of rectilinear images forms a substantially linear projectionof the GI tract.
 10. The method of claim 9, further comprising:dynamically displaying, by the image display system, the substantiallylinear projection of the GI tract in a manner dictated by a humanoperator.
 11. The method of claim 8, further comprising: permitting, bythe image display system, a human operator to input a comment related toa portion of the GI tract in the composite image.
 12. The method ofclaim 8, further comprising: causing, by the image display system,graphical user interface (GUI) controls to be displayed that aremanipulated by a human operator to change a display of the compositeimage so as to focus on a particular feature of the GI tract.
 13. Themethod of claim 8, wherein the converting comprises: converting theplurality of annular images to a plurality of trapezoidal images; andconverting the plurality of trapezoidal images to a plurality ofrectangular images.
 14. The method of claim 8, wherein the storingcomprises: storing specific location information with respect to areference point for the plurality of annular images corresponding to theplurality of portions of the GI tract.